Search for: All records

Transformer models have revolutionized machine learning, yet the underpinnings behind their success are only beginning to be understood. In this work, we analyze transformers through the geometry of attention maps, treating them as weighted graphs and focusing on Ricci curvature, a metric linked to spectral properties and system robustness. We prove that lower Ricci curvature, indicating lower system robustness, leads to faster convergence of gradient descent during training. We also show that a higher frequency of positive curvature values enhances robustness, revealing a trade-off between performance and robustness. Building on this, we propose a regularization method to adjust the curvature distribution and provide experimental results supporting our theoretical predictions while offering insights into ways to improve transformer training and robustness. The geometric perspective provided in our paper offers a versatile framework for both understanding and improving the behavior of transformers. 

Despite several known idiosyncrasies separating the synchronous and the asynchronous models, asynchronous secure multi-party computation (MPC) protocols demonstrate high-level similarities to synchronous MPC, both in design philosophy and abstract structure. As such, a coveted, albeit elusive, desideratum is to devise automatic translators (e.g., protocol compilers) of feasibility and efficiency results from one model to the other. In this work, we demonstrate new challenges associated with this goal. Specifically, we study the case of parallel composition in the asynchronous setting. We provide formal definitions of this composition operation in the UC framework, which, somewhat surprisingly, have been missing from the literature. Using these definitions, we then turn to charting the feasibility landscape of asynchronous parallel composition. We first prove strong impossibility results for composition operators that do not assume knowledge of the functions and/or the protocols that are being composed. These results draw a grim feasibility picture, which is in sharp contrast with the synchronous model, and highlight the question: Is asynchronous parallel composition even a realistic goal? To answer the above (in the affirmative), we provide conditions on the composed protocols that enable a useful form of asynchronous parallel composition, as it turns out to be common in existing constructions. 

Abstract BackgroundThe blowflyLucilia cuprinais a destructive parasite of sheep that causes flystrike or myiasis. Larvae consume the animal’s living flesh, producing large wounds that can lead to death. The main aim of this study was to identify genes that may play important roles in the behavior and physiology ofL. cuprinalarvae. MethodsAn RNA-Seq analysis of RNA from whole larvae at different developmental stages and third-instar head and gut tissues was used to identify sensory receptors and other genes relevant to the physiology ofL. cuprinalarvae. In addition, CRISPR/Cas9 gene editing was used to obtain a loss-of-function mutation for theL. cuprinaodorant coreceptor gene (LcupOrco). The response of mutant larvae and adult females to fresh and rotten meat at different temperatures was evaluated. ResultsThe RNA-Seq analysis suggested that odorant (OR), gustatory, ionotropic, andPickpocketreceptors may not play a central role in theL. cuprinalarval sensory signaling and digestive systems. Rather, ATP-binding cassettes (ABCs) were highly enriched in head and gut RNA, and odorant-binding proteins (OBPs) only in the head. To confirm that ORs are not essential for larval detection of rotten beef, diet-choice assays were performed including larvae and adults homozygous for a null mutation inLcupOrco. While the attraction of adult females to rotten beef was disrupted,LcupOrcomutant larvae showed no change in diet preference. ConclusionsThe expression pattern of the ABC and OBP gene families suggests a central role in the sensory system of theL. cuprinalarva for these receptors. Behavioral assays showed that ORs are essential for the adult female response to rotten beef, but not for larval behavior. These findings are consistent with high levels of expression ofLcupOrcoin the adult female antenna but very low expression in larvae. Graphical abstract